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Abstract 

In this work we give a proof of universality with respect to the choice of 
the statistical distribution of the quenched noise, for mean field bipartite 
spin glasses. We use mainly techniques of spin glasses theory, as Guerra's 
interpolation and the cavity approach. As a direct conseguence of our 
results, we have a proof of Lao convergence of the free energy of the 
Hopfield Model to its expectation value. 

Introduction 

Although spin glass models, expecially the well known Sherrington Kirkpatrick 
(SK) model, have been largely investigated in the past years [H] [12] [18] , limited 
attention has been payed for bipartite systems. Despite that, bipartite spin 
glasses have a very interesting matematical structure, quite similar in several 
aspects to the Hopfield Model for neural networks [II] [3] H], and find many 
applications in modelling (see [S] and references therein). 

The problem of universality is an important aspect of the theory of random 
matrices and very recently new general results have been achieved |21j . 

From the spin glass perspective, universalty is usually considered not of primarly 
importance, since it is believed that the main characteristics of spin glasses are 
independent on the choice of the distribution of quenched noise. It is infact the 
case of the SK Model. On the other hand, Guerra's interpolation, that is the 
key ingredient of the proof of the Parisi formula by Talagrand [II][in], works 
only with gaussian random interaction (it is based on the Wick rule). Therefore, 
the problem to justify the particular choice of gaussian interaction in SK model 
was dealt and solved [T7][5)[g]. 

The same problem arises in dealing with bipartite spin glasses: a systematic 
mathematical study of such a model, by interpolation method, has been started 
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in [5], motivated by a well known analogy with the Hopfield Model |18|[7|[5][i]. 
that is infact a special bipartite spin glass. 

It will be clear from what follows that, even though the mathematical structure 
of bipartite spin glasses and the Hopfield Model is the same for many aspects, 
their universality properties are quite different. In particular, although we have 
no complete proof of that, we believe that universality does not hold for the 
Hopfield Model, at least not in the whole phase diagram. 
The work is organized as follows: 

In Section 1 the general structure of bipartite models is given, even in the case 
of systems of (bounded) soft spin, although in the whole paper only dichotomic 
variables are used (the generalization is straightforward) . 

In Section 2 we will state the results: the pressure of the bipartite spin glass 
model, with certain conditions on the noise, is close in distribuition to the one 
with gaussian noise, when the size of the system grows to infinity. Then we 
state that their are close also in Lp norm for some p to be specified later. 
Proofs of the statements are given in Section 3. 

Lastly, in Section 4 we will point out the results that can be extended to the 
Hopfield Model, in particular we give a proof of the Loo convergence of the 
pressure to the quenched one. Furthermore, we will discuss the points where 
our strategy fails for such a model, trying to give some explanations. 



1 The Structure of Bipartite Models 

We will deal with a set of N i.i.d. random spin variables Ci, « = 1, ---jN, with 
any probability distribution /i(cr), symmetric with compact support [—L, L]. In 
particular we notice that ECT[cr] — 0, and for a given bounded function of spin 
/(cr), we must have E^[/(cr)] < L[sup^^[_L,L] /(o-)]- 

Let us consider now for every N another set of i.i.d. random spin variables r^, 
/X = 1, ■■■,K, with any probability distribution vir) with the above properties, 
but in general /i(cr) and v{t) may be different. Therefore we have two distinct 
sets (or parties hereafter) of different spin variables, and we let them interact 
via the hamiltonian: 



where the are also i.i.d. r.v., with E[^] ~ and = 1, i.e. the quenched 
noise ruling the mutual interactions between parties. It is then defined a mean 
field bipartite spin glass model [5J. 

For sake of simplicity, in what follows we deal with both parties formed by 
dichotomic variables (as usual, sums denote not normalized expectations). 
The partion function, the pressure and the free energy of the model are defined 
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as 



Zn,k{P,0 = ^CXp(-/3iJAr,A'(C;a,T)), (2) 



a.r 



AnMP) = jif^^^^ogZ^MP). (3) 

InAP) = -'^AnMP)- (4) 

We can define also the Boltzmann state for a generic function of the spin vari- 
ables F((T, r): 



WAT, 



k{F) = Z^]j,{p) [Fia, t) exp r))] . 



The main goal of the theory is the control of the free energy in the thermo- 
dynamic limit, i.e. for N,K — > oo, when the size of the two parties grows to 
infinity, such that N/{N + K) ^ a e {0, 1) and K/{N + K) ^ {1 - a) e (0, 1). 
We adopt this latter definition of thermodynamic limit, and thus the thermody- 
namic functions depend by the additional parameter a, ruling the relative ratio 
between the parties: 

lim AN.Kil3)^A{a,P). 

N,K ' ^' ' 

At the moment no rigorous proof of the existence of such a limit is known. 



2 Results 

We claim that the free energy of bipartite spin glass models is universal with 
respect to the choice of the statistical distribution of the quenched noise. 
In order to give a rigorous proof of this assertion, we need to recall the following 
result [H]: 

Lemma. Let ^ a real r.v. such that, if g is a unit centered gaussian r.v., it 
is EgC/'' = lE^^'", V/c = 1...TO, and E[|^™+^|] is finite. Furthermore he f a real 
function in C™, such that ||/(™)||oo = sup l/^™)] < oo. Then 

mim-nennm < (^) mr+'m^^^iu (5) 

This lemma gives the error we make when we use the gaussian derivative rule 
for functions of random variables close to be gaussian up to order m. Since 
we want to use Guerra's interpolation technique, it is crucial to have such a 
lemma that compares derivatives. For example, in the Lindeberg approach, this 
is replaced by an integral analogue (see Theorem 1.1 in [9]). The two methods, 
even though the Lindeberg approach gives a slightly weaker condition on the 
random interaction, seem to be equivalent: they give similar estimates, and 
they both depend in a crucial way on the behaviour of the derivatives of the 
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Boltzmann mean value of the spin part of the internal energy (that is, cj((Tir^), 
or, in the notation of [9], a;(A2)). 

If we take a random matrix , each entry with the hypothesys of the previous 
lemma, we define the pressure of the ^-noise model as 



1 



N+K 



N + K 



log^exp 13a 



N + K 



N,K 



(6) 



while we set 



N 



l^log^exp L^JZKlY^g^^^^r, 

cr,r \ ^i— 1 



(7) 



with gij normal distribuited. So we can prove our first result: 



Theorem 1. Let ^ be a real r.v., as in the hypothesis of the previous Lemma, 
with m > 2, and g be a unit centered gaussian. Then we have 



E, 



9:? 



(8) 



That is completely analogous with the achievement obtained in 18J[9J for the 
Sherrington-Kirkpatrick model. 

Now we have to evaluate fluctuations. We will use an argument based on the 
cavity technique in order to state the following 

Theorem 2. BeS^tj the entries of a random matrix such that, for a fixed m > 0, 
E[|^|P] is bounded for every positive real number p < m. Then 



E 



|4 



N,K 



A 



N.K\ 



< a 



(9) 



(N + K) 

with Cp an universal constant, depending only by p. 

The combination of the two theorems gives immediately the following 

Corollary 1. In the hypothesis of the Lemma, for a fixed m > 2 and a positive 
p < m + 1 we have 



E„ 



/ 1 



I iVP/2 



(10) 

(11) 
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3 Proofs 



Proof of the Lemma. If we expand in Taylor series both the function / and its 
derivative f^^\ we get 



/(i)(^)_/(i)(0)_e/(2)(0)-., « 



(m - 1)! 

m-2 



(0) 

;(™-i)(0) 



to! 

^m— 1 



< 



(to - 1)! 



(to -2)! 

Now, since the mean value of ^ is zero, we have 

Hum - ni'm'm I = - im] - nemfm \ 



11/^'^ 



(12) 



Due to the given relation between the to gaussian moments and the ones of 
^, we can add and subtract analogous terms in ((T2|) . according to the Taylor 
expansions of / and /^^': 



iE[e(/(o-El/^'^^(o)] 



i=0 



i=l ^ 



< E 



< 



< 



< 



E[|^|"+i 



/'(?) - E 



-/«(0) 



to 

||/<"^||o 

to! 

TO + 1 



■11/^' 



TO 



, ll/^-^lloc+EK^ . 
' (to — Ij! 



□ 



Proof of Theorem 1. It is useful to introduce the interpolating partition func- 
tion: 



ZN+K{t) = E^.r exp /3 



2t 



N + K 



and the interpolating pressure 

AN+K{t) = E^,glogZw+/f(t). 
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It is easily seen that An+k{0) = EgA^j^_^_^ and An^k{^) = ^^^n+k- Further- 
more, in virtue of the previous lemma, we have 



and since 



(13) 



bearing in mind the Lemma, we get 



ij ij 



Then we easily have 
d 



dt 



AN+K{t) 



Now we have to estimate the m-th derivative of the state u)t{o'iTj) with respect to 
^. If we name P'" the cumulant polynomial of degree m in (cTiTj) with respect 
to the Gibbs measure (for example = ijj{{aiTj)^) — 3uj{{aiTj)'^)u!{aiTj) + 
2uj^{aiTj), see for instance pp: the coefhncients are infact the same of Aizenmann 
Contucci relations in the SK model), it is easy to check that 



(m) _ 



l3V2i) (Ar + if)-"/2pm+i 



hence (for dichotomic spin variables) 

||w^'"^(CT.r,)|| < i^l3V2ij [N + K)-"'^^im + 1)!. 
Therefore, it follows that 



(14) 



(15) 



E 



— 



< 



< 



dt 



-AN+K{t) 



(%/2^)™+ia(l - a) /-^^^ tt"-!)/^ 



(iV + if)(™-i)/2 7o 



E[ 



im+ll 



im+ll 



and the theorem is proven. 



□ 
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Proof of Theorem 2. In primis we construct an increasing sequence of subsets in 
the set of spin variables. Given a NK-svze system, we cover it with sets labeled 
by /i, such that for each h there is a subset of size NhKh < NK, and when h 
increases by one, we have alternatively either N^+i = Nh + 1, Kh+i = Kh, or 
Nh+i = Nh, Kh+i = Kh + 1, h = 0,..., NK. Furthermore we have Nq = Kq ^ 
and Np^K = N, K^k = K . This sequence induces a natural filtration on the 
^ variables, Fh = {^i, ^/i}, i.e. the noise occurring to define a NhKh-sae 
bipartite system. 

Then we proceed with a cavity argument. We can write the hamiltonian of 
the system as the hamiltonian of a smaller one, with suitable additional terms, 
namely the cavity field. This is a standard approach in spin glasses |16]|18j. 
Thus, bearing in mind our sequence, we write 



N,K 



N 

K I :^ N K 



N + K 



where I3h - P^{Ni, + Kh)/{N + K), an. = Nh/{Nh + Kh), as usual h^r) = 

'^f=i ^^'^ y {a) = Nf^ ^re the cavity fields, and we 

indicate with a bar the spin and the noise variable outside of the h-th sequence 
and filtration. Hence we have 



l0gZjv,_K(/3) = log ZN„,K,XPh) 

+ logwjv^,A'^ Eff„^^,...ff„]E^^^^i...^^.exp 



N 



i=Nh + l 



K 



PhV^^h Y ^]h^{(^)+P 

3=K^ + l 



N 



N- 



■ E E 4 

i=Nh + l]=KH + l 



Now we can define the function 



- E- 



exp 



AT 



phV^i^-ah) Y ^^^'^^^ 

i=Nh + l 



K 



Ph^h Y rjh^a)+(3 



N + K 



N 



K 



Y E ^^3^i 



that is a suitable generalization of the well known cavity function introduced 
in spin glass theory many years ago [lOj. It is worthwhile to notice that V'o = 
log AT, if, and ^NK = 0. So we have that 

¥.[\ogZMMFh] = {Nh + Kh)AN,,K, + M^h). 
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Now, following [S], we can introduce the martingale 

= E[log ZN,K\Fh] ~ E[logZw,A'|i^/.-i] 

and notice that 



^ NK 



N + K 



h=l 



\A%,k~An,k\ 



(16) 
(17) 



At this point we need a bound on Afi: 

|Ah| = \^ph - iph-i + '^og Zn,,^k,,- log Zn^^^^k^^,] 
= logw* (e^V^i"''''^ < I3y 

hence 



N + K 



(18) 



(19) 



N + K' 

Here w* denotes the expectation value with respect to an ausiliar Gibbs measure, 
with weights 



AT 



exp -PhHN^_^.KH-i + Ph\/'2.{1 - OLh) ^ (Tih\T) 

V i=Nh + l 



K 



N + K 



N K 



J2 ^'3''' 

i=N,, + lj=Kh + l 



Now we use a martingale moments inequality (see for instance [15j), in order to 
state 



E 



^ NK 

— Fa 



N + K 



h=l 



< CpE 



NK 



(N + K)^^^^^ 



p/2 



(20) 



with Cp an universal constant independent on ^. Therefore, putting together 
(dZD, (HH) and (EOl): 



E 



□ 



4 Further Remarks and Open Problems 

In this work we have established the invariance of the free energy of bipartite 
spin glass, with respect to the choice of the statistical distribution of the random 
interaction in thermodynamic limit. Our method is on the same line of [S], and 
anyway both are based on Guerra's interpolation and cavity field approach. Our 
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results can be interpreted as follows: at first we have shown that, under certain 
hypothesis on the random interaction, the free energy converges in distribution 
to the one with gaussian interaction (Theorem 1); then we have shown that it 
converges also in Lp, for suitable values of p (Theorem 2). Of course all that 
holds provided the proof of the existence of the thermodynamic limit of the free 
energy for bipartite models of spin glasses, that remains an open problem. 
We can make some other consideration. At first we notice that in bipartite 
models with ±1 spin, we can improve estimate ([T5|) . in order to get faster rate 
of convergence, but for less values of p: 

log..* (e^V^f'^'^^) < P'j;^e (21) 

This is convenient in the models of interest, where the random interaction has 
all moments bounded {e.g. ±1, gaussian). So we can give an other version of 
Theorem 2: 

Theorem (2 v2). If we deal with dichotomic spin variables, in the hypothesis 
of Theorem 2 but for < p < (to + l)/2, we have 



E 



with Cp an universal constant, depending only by p. 

The last theorem holds also if only one of the party is made by ±1 spin, and 
the other is formed by soft bounded spin. Anyway we have the following 

Corollary 2. Consider a model of bipartite spin glass with spin ±1. Pro- 
vided the thermodinamic limit for the free energy exists, we have that Vp > 

■^N K — ^ ^(q^: /3) (gaussian random interaction) and — ^ A{a, (5) (di- 
chotomic random interaction). Furthermore, in both cases and Vp > 0, the 

convergence rate is max (^O (^pyTFIp) ' \^n,k{P) — A{a, /3)|p^ . 

For unbounded spin the model is the most interesting: if we take a bipartite 
model with ±1 and K gaussian spin, interacting via the hamiltonian 

ill 

this turns out to be equivalent to the Hopfield Model for neural network |18|[7]. 
We notice that the normalization is different to the one used in this paper, 
hence now a — K/N . Originally the model was introduced with ±1 random 
interaction. Later also its gaussian version has been studied jl4)[l]. 
Thus we would like to extend in part our results to this model; unfurtunately 
this seems very hard to do. The main difficulty is in Theorem 1: in both the 
approaches for proving universality in spin glasses, the one based on the Wick 
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rule, or interpolation method, ([TTj, [S], ours), or the one based on Lindeberg 
argument [9], it is crucial to get an estimate on the derivatives of a;(tTiT^); that 
is trivial in our model, or even in SK model, but it has a deep significance in 
the Hopfield Model. Actually we know that universality (in a strict sense) in 
the Hopfield Model does not hold: it sufiices to match the results one can get 
for the model with gaussian or dichotomic interaction Infact in general 

we have 



and furthermore the models are of course different in the limit a — [13] . 

So a sharp estimate on the derivatives of u{aiT^) would give a region where 

universality is expected. This is anyway an open problem. 

On the other hand, we can easily extend Theorem 2 (better in its second version, 

performing an integration over gaussian variables in order to get (I2ip ) to the 

Hopfield Model, with both gaussian or dichotomic interaction: 

Theorem (2 for the Hopfield Model). In the hypothesis of Theorem 2 hut for 
< p < (m + l)/2, we have 



with Cp an universal constant, depending only by p. 
and we get immediately the corollary 

Corollary 3. Provided the thermodinamic limit for the free energy of the Hop- 
field Model exists, for both the models with gaussian or ±1 random interaction. 



Of course the proof of the existence of the thermodynamic limit for the free 
energy of the Hopfield Model is as hard as in the bipartite spin glass. But for 
the original Hopfield Model we can do even better, proving convergence in the 
sup norm. Infact by ([TO]) and (PT|) we are sure that the difference between the 
pressure and its expectation value is bounded uniformly in ^: 






sup \A%i,{0 



AnM < sup 



?=±i 



a/32. 



Furthermore 
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so 



\im\\A% - An,k\\ 

n ' 



P 



= \\A,k-An,k\\ 



GO 




(22) 



since, for p > 1, Cp < Cp for a certain costant C > [T3]. Thus as a direct 
conseguence of our work we have the following final 

Theorem 3. Provided the existence of the thermodynamic limit for the Hopfield 
Model, we have that the pressure converges in Loo norm to its mean value, with 
rate 



This result is in the same wake of the ones in pf]|18|. 
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